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Abstract. In this research, students’ sentiments and emotions embedded in their 
learning journals are analyzed to understand their attitudes to mobile-based lessons 
as they progress during an English as a Foreign Language (EFL) course. Sentiment 
Analysis (SA) was utilized to extract emotions and sentiment throughout students’ 
learning experience, as expressed in their weekly online learner journals. The 
sentiment scores were generated from four sentiment dictionaries with different 
scales. The findings suggest that overall, the students had a positive sentiment and 
emotions toward mobile learning, consisting of anticipation, trust, joy, and surprise. 
The strongest negative emotion was fear, which may be explained by anxiety 


surrounding communication in a foreign language. 
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1. Introduction 


Emotional obstacles impede learning (Pekrun, Goetz, Titz, & Perry, 2002; Zeidner, 
2014), as students typically learn and perform better when they experience 
positive sentiments about a subject and learning context. In this research, students’ 
sentiments and emotions embedded in their learning journals are analyzed to 
understand their attitudes to mobile-based EFL. SA was utilized to extract emotions 
and sentiment during the course, as expressed in students’ weekly online learner 
journals. Sentiment scores were generated from four sentiment dictionaries with 
different scales, including syuzhet, bing, afinn, and nrc. The findings suggest that 
overall, the students had positive sentiment and emotions toward mobile learning as 
defined by these terms: anticipation, trust, joy, and surprise. The strongest negative 
emotion was fear, which may be explained by anxiety surrounding communication 
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in a foreign language. SA has been effective for evaluating mobile pedagogical 
affordances (Bano, Zowghi, & Kearney, 2017). However, the focus has been on 
consumer sentiment toward mobile social networks, with little coverage of mobile 
learning environments (Abdulsalami et al., 2017; Hew, Hu, Qiao, & Tang, 2020; 
Martin, Ortigosa, & Carro, 2012; Rani & Kumar, 2017). 


2. Method 


The study design was a case study adopted for one academic year to gain a deeper 
understanding of the outcomes of completing collaborative learning activities 
through mobile devices by four Japanese university undergraduate EFL classes on 
translation. Each class formed one case study group with between five and eight 
members. All factors remained constant across the groups, and participation in the 
study was voluntary. The groups were comprise of: 


¢ Group I: five female and two male students; 
¢ Group 2: eight female students; 

¢ Group 3: six female students; and 

¢ Group 4: six female students. 


The data collection for each group was identical and consisted of each student 
submitting an open-ended e-journal at the end of each week in the students’ L1, 
Japanese, with comments on any use of mobile devices for homework activities. 
These e-journals were then translated into English by the researcher before the 
analysis. While translation is always a limitation in research, there were no 
significantly complex word meanings that could strongly influence the result. 
However, this is a topic that may be discussed further in an extended version 
of this paper. The SA was performed using the R package syuzhet (Manning et 
al., 2014) for sentiment scores and emotion classification. All other text mining 
was done through the tm R package (Feinerer & Hornik, 2019). Sentiments are 
classified as positive, neutral, or negative; and numerical. The syuzhet package 
was used for generating sentiment scores and has four sentiment dictionaries 
with different scales, including syuzhet, bing, afinn, and nrc. As explained in 
Mhatre (2020, n.p.), sentiment scores using the syuzhet take the form of a decimal 
range from -1 (most negative) to +1(most positive), bing is a binary scale with 
-| indicating negative and +1 indicating positive sentiment, and similarly, afinn 
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is an integer scale ranging from -5 to +5. However, the nrc emotion lexicon 
(Mohammad & Turney, 2013) is a list of English words and their associations 
with eight basic emotions (anger, fear, anticipation, trust, surprise, sadness, joy, 
and disgust) and two sentiments (negative and positive) manually annotated 
through crowdsourcing. 


3. Results and discussion 


The e-journals’ data set of 3,800 English words was a word frequency table. 
To identify themes, the top five most frequent words in the text were identified: 
‘homework’ (61), ‘phone’ (51), ‘use’ (44), ‘mobile’ (41), and ‘can’ (25). The 
two most frequent words are ‘homework’ and ‘phone’, which is not unexpected 
considering the activities. Also, the two words ‘use’ and ‘can’ imply positive 
agency. To give more insight into the context, correlation (corlimit=0.25) is used to 
identify which words appear most often with these five most frequently occurring 
words. For ‘homework’, ‘look’, ‘read’, and ‘rememb’ (the root of remember) all 
appear together 76% of the time. The negative word roots that commonly appeared 
with ‘homework’ were ‘embarrass’ (57%), ‘grumbl’ (57%), and ‘negat’ (49%), 
while the positive word roots were ‘good’ (47%), ‘desir’ (38%), ‘happi’ (38%), 
and ‘nice’ (28%). This stronger negative association with homework may not be 
a surprise, but ‘anytime’ (37%) and‘anywhere’ (26%) suggest that the students 
noticed these well-known mobile device affordances. 


The next two frequent words of interest are ‘phone’ and ‘mobile’, and they may 
refer to the same object — mobile phone. The word root most associated with both 
words is ‘busi’, at 74% for ‘phone’ and 63% for ‘mobil’, suggesting that they 
associate this device to some extent as a tool. The word ‘phone’ correlates with no 
negative words, but the positive word roots are ‘happi’ (67%), ‘advantage’ (57%), 
‘comfort’ (57%), ‘thank’ (49%), ‘great’ (34%), ‘nice’ (33%), and ‘good’ (28%). 
Also, the correlation of ‘street’ (57%), ‘technolog’ (57%), and ‘walk’ (56%) may be 
explained by the requirement that students collect examples during their everyday 
lives outside of school to complete the homework activities. Likewise, ‘mobil’ 
is correlated with ‘happi’ (54%), ‘amaz’ (36%), ‘conv’ (convenient) (36%), “bett’ 
(36%), ‘benefit’ (36%), ‘comfort’ (36%), ‘nice’ (29%), and the negative ‘negat’ 
(36%). These results strongly indicate that the students have a generally positive 
view of their mobile phone in these homework activities. 


Table | includes the syuzhet package-generated sentiment scores using the four 
sentiment dictionaries, and the normalized scores used for comparison. 
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Table 1. Syuzhet generated sentiment scores 


Min. 1st Qu. |Median |Mean (|3rd Qu. |Max. | Normalized 
syuzhet | -0.50 0.80 1.30 1.45 1.81 6.00 1,1, 1,1,-1,1 
bing -1.00 0.00 1.00 1.16 2.00 7.00 1, 1,0, 1,0, 1 
afinn -2.00 0.00 2.00 2.23 3.25 12.00 /1,1,1,1,-1,1 


The nrc function, in line with Mhatre (2020), “returns a data frame with each row 
representing a sentence [and] ten columns, one for each of the eight emotions and 
one column for positive sentiment valence and one for negative sentiment valence” 
(n.p.). Figure | shows the number of instances of words in the text associated 
with these eight emotions. Positive words associated with ‘anticipation’ occur 
65 times and may be explained by the students’ comments that they had never 
done this type of learning activity using mobile phones. The next positive words 
relate to ‘trust’ (40), then ‘joy’ (18), and ‘surprise’ (11). The strongest negative 
emotion words relate to ‘fear’ (38), which may be explained by their general fear 
of communicating online in their second language, as expressed in their journals. 
This is followed by the negative emotion of ‘sadness’ (9), ‘anger’ (6), and ‘disgust’ 
(4). Figure 2 compares the number of emotion words as a percentage of the total of 
meaningful words. The positive emotion ‘anticipation’ accounts for over 30% of 
all meaningful words in the e-journals. Moreover, ‘trust’ is second, at over 20% of 
meaningful words. They are followed by the negative emotion ‘fear’, also at 20%. 
Overall, the words associated with positive emotions that the software considers 
meaningful account for over 65% of total words. 


Figure 1. Number of words associated with eight main emotions 
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Figure 2. Words associated with each sentiment as a percentage 
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4. Conclusions 


The findings suggest that students in this study held positive sentiment on the use 
of mobile devices for collaborative activities, as shown by the syuzhet, bing, and 
afinn results. The nrc result indicates that these sentiments can be best described as 
‘anticipation’, ‘trust’, ‘joy’, and ‘surprise’. At the same time, they had the emotion 
of ‘fear’ for the activities, which could be related to the anxiety of communicating 
in a foreign language. However, this requires more research to clarify the possible 
connection. 


5. Acknowledgments 


This research was supported by JSPS KAKENHI Grant Number JP20K00862. 


118 


Sentiment analysis of students’ attitudes toward mobile learning activities 


References 


Abdulsalami, A., Ahmad, B., Umar, M., Abubakar, A., Jauro, F., Kufena, A., & Ekoja, E. (2017). 
Sentiment analysis of students’ perception on the use of smartphones: a cross sectional study. 
In 2017 Second International Conference on Informatics and Computing (ICIC). https://doi. 
org/10.1109/iac.2017.8280625 

Bano, M., Zowghi, D., & Kearney, M. (2017). Feature based sentiment analysis for evaluating 
the mobile pedagogical affordances of apps. In JFIJP World Conference on Computers in 
Education. https://doi.org/10.1007/978-3-3 19-743 10-3_ 30 

Feinerer, I., & Hornik, K. (2019). tm: text mining package [Version R package version 0.7-7]. 
https://CRAN.R-project.org/package=tm 

Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: 
a gradient boosting trees supervised machine learning and sentiment analysis approach. 
Computers & Education, 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724 

Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S., & McClosky, D. (2014). 
The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual 
meeting of the association for computational linguistics: system demonstrations. https://doi. 
org/10.3115/v1/p14-5010 

Martin, J. M., Ortigosa, A., & Carro, R. M. (2012). SentBuk: sentiment analysis for e-learning 
environments. In 2012 International Symposium on Computers in Education (SIE). 

Mhatre, S. (2020 May 13). Text mining and sentiment analysis: analysis with R. redgate. https:// 
www.red-gate.com/simple-talk/sql/bi/text-mining-and-sentiment-analysis-with-1/? 

Mohammad, S. M., & Turney, P. D. (2013). Crowdsourcing a word—emotion association lexicon. 
Computational Intelligence, 29(3), 436-465. 

Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic Emotions in Students’ Self- 
Regulated Learning and Achievement: A Program of Qualitative and Quantitative Research. 
Educational psychologist, 37(2), 91-105. https://doi.org/10.1207/s15326985ep3702_4 

Rani, S., & Kumar, P. (2017). A sentiment analysis system to improve teaching and learning. 
Computer, 50(5), 36-43. 

Zeidner, M. (2014). Anxiety in education. In R. Pekrun & L. Linnenbrink-Garcia (Eds), 
International handbook of emotions in education (pp. 265-288). Taylor & Francis. 


119 


esearch-publishing.net 


Published by Research-publishing.net, a not-for-profit association 
Contact: info@research-publishing.net 


© 2020 by Editors (collective work) 
© 2020 by Authors (individual work) 


CALL for widening participation: short papers from EUROCALL 2020 
Edited by Karen-Margrete Frederiksen, Sanne Larsen, Linda Bradley, and Sylvie Thouésny 


Publication date: 2020/12/14 


Rights: the whole volume is published under the Attribution-NonCommercial-NoDerivatives International (CC BY-NC-ND) 
licence; individual articles may have a different licence. Under the CC BY-NC-ND licence, the volume is freely available 
online (https://doi.org/10.14705/rpnet.2020.48.9782490057818) for anybody to read, download, copy, and redistribute 
provided that the author(s), editorial team, and publisher are properly cited. Commercial use and derivative works are, 
however, not permitted. 


Disclaimer: Research-publishing.net does not take any responsibility for the content of the pages written by the authors 
of this book. The authors have recognised that the work described was not published before, or that it was not under 
consideration for publication elsewhere. While the information in this book is believed to be true and accurate on the date of 
its going to press, neither the editorial team nor the publisher can accept any legal responsibility for any errors or omissions. 
The publisher makes no warranty, expressed or implied, with respect to the material contained herein. While Research- 
publishing.net is committed to publishing works of integrity, the words are the authors’ alone. 


Trademark notice: product or corporate names may be trademarks or registered trademarks, and are used only for 
identification and explanation without intent to infringe. 


Copyrighted material: every effort has been made by the editorial team to trace copyright holders and to obtain their 
permission for the use of copyrighted material in this book. In the event of errors or omissions, please notify the publisher of 
any corrections that will need to be incorporated in future editions of this book. 

Typeset by Research-publishing.net 


Cover theme by © 2020 Marie Flensborg (frw831@hum.ku.dk), based on illustration from freepik.com 
Cover layout by © 2020 Raphaél Savina (raphael@savina.net) 


ISBN13: 978-2-490057-81-8 (Ebook, PDF, colour) 


British Library Cataloguing-in-Publication Data. 
A cataloguing record for this book is available from the British Library. 


Legal deposit, France: Bibliotheque Nationale de France - Dépét légal: décembre 2020. 


